{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ten Steps to Keras\n",
    "\n",
    "### Deep Learning Tutorial using Keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div>\n",
    "    <img style=\"text-align: left\" src=\"imgs/keras-tensorflow-logo.jpg\" width=\"40%\" />\n",
    "<div>\n",
    "\n",
    "<div>\n",
    "    <img style=\"text-align: left\" src=\"imgs/conference_logo.png\" width=\"40%\" alt=\"EuroScipy 2017 Logo\" />\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get the Materials"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"imgs/github.jpg\" />\n",
    "\n",
    "#### Right Now\n",
    "\n",
    "```shell\n",
    "\n",
    "git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git -b euroscipy2017\n",
    "```\n",
    "\n",
    "\n",
    "#### Later\n",
    "\n",
    "```shell\n",
    "\n",
    "git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git\n",
    "git checkout tags/euroscipy2017\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Goal\n",
    "\n",
    "- **Introduce** main features of Keras APIs to build Neural Networks.   \n",
    "- **Learn** how to implement simple and complex Deep Neural Networks Architectures using Keras.   \n",
    "- **Discover** Keras Implementation and Internals.\n",
    "\n",
    "\n",
    "- **Note**: examples and hands-on exercises will be provided along the way. This means that you should be able to run (almost) everything on your laptop without needing GPU(s). Even if, with GPUs code run faster :P"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Outline (in ten-ish Notebooks)\n",
    "\n",
    "1. _Multi-layer Fully Connected Networks (and the `backends`)_\n",
    "2. _Hidden Layers features and Embeddings_\n",
    "3. _Convolutional Networks_\n",
    "4. _Hyperparameter Tuning_\n",
    "5. _Cutsom Layers_\n",
    "6. _Deep CNN and Residual Networks_\n",
    "7. _Transfer Learning and Fine Tuning_\n",
    "8. _Recurrent Neural Networks_\n",
    "9. _AutoEncoders_\n",
    "10. _Multi-Modal Networks_\n",
    "\n",
    "- - - -"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Requirements"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial requires the following packages:\n",
    "\n",
    "- Python version 3.5\n",
    "    - Python 3.4 should be fine as well\n",
    "    - likely Python 2.7 would be also fine, but *who knows*? :P\n",
    "    \n",
    "\n",
    "- `numpy` version >= 1.12: http://www.numpy.org/\n",
    "- `scipy` version >= 0.19: http://www.scipy.org/\n",
    "- `matplotlib` version >= 2.0: http://matplotlib.org/\n",
    "- `pandas` version >= 0.19: http://pandas.pydata.org\n",
    "- `scikit-learn` version >= 0.18: http://scikit-learn.org\n",
    "- `keras` version >= 2.0: http://keras.io\n",
    "- `tensorflow` version 1.2: https://www.tensorflow.org\n",
    "- `ipython`/`jupyter` version >= 6.0, with notebook support\n",
    "\n",
    "(Optional but recommended):\n",
    "\n",
    "- `hdf5` and `h5py` (required if you use model saving/loading functions in keras)\n",
    "- **NVIDIA cuDNN** if you have NVIDIA GPUs on your machines.\n",
    "    [https://developer.nvidia.com/rdp/cudnn-download]()\n",
    "\n",
    "The easiest way to get (most) these is to use an all-in-one installer such as [Anaconda](http://www.continuum.io/downloads) from Continuum. These are available for multiple architectures."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up the Environment\n",
    "\n",
    "If you need support to setup and replicate the environment with all the required library, please\n",
    "find all the step-by-step instructions in the [README.md](README.html) file.\n",
    "\n",
    "This repo ships with `.yml` files to replicate the Python Conda environment (on both Linux and OSX).\n",
    "\n",
    "Moreover, helpful tips on how to setup theano and tensorflow with GPU support are also provided"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python Version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I'm currently running this tutorial with **Python 3** on **Anaconda**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python 3.5.4 :: Anaconda custom (64-bit)\r\n"
     ]
    }
   ],
   "source": [
    "!python --version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Configure Keras with tensorflow\n",
    "\n",
    "1) Create the `keras.json` (if it does not exist):\n",
    "\n",
    "```shell\n",
    "touch $HOME/.keras/keras.json\n",
    "```\n",
    "\n",
    "2) Copy the following content into the file:\n",
    "\n",
    "```\n",
    "{\n",
    "    \"epsilon\": 1e-07,\n",
    "    \"backend\": \"tensorflow\",\n",
    "    \"floatx\": \"float32\",\n",
    "    \"image_data_format\": \"channels_last\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\r\n",
      "    \"image_data_format\": \"channels_last\",\r\n",
      "    \"epsilon\": 1e-07,\r\n",
      "    \"floatx\": \"float32\",\r\n",
      "    \"backend\": \"tensorflow\"\r\n",
      "}\r\n"
     ]
    }
   ],
   "source": [
    "!cat ~/.keras/keras.json"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test if everything is up&running"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Check import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import scipy as sp\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import keras"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Check installeded Versions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "numpy: 1.12.1\n",
      "scipy: 0.19.1\n",
      "matplotlib: 2.0.2\n",
      "iPython: 6.1.0\n",
      "scikit-learn: 0.19.0\n"
     ]
    }
   ],
   "source": [
    "import numpy\n",
    "print('numpy:', numpy.__version__)\n",
    "\n",
    "import scipy\n",
    "print('scipy:', scipy.__version__)\n",
    "\n",
    "import matplotlib\n",
    "print('matplotlib:', matplotlib.__version__)\n",
    "\n",
    "import IPython\n",
    "print('iPython:', IPython.__version__)\n",
    "\n",
    "import sklearn\n",
    "print('scikit-learn:', sklearn.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "keras:  2.0.8\n",
      "Theano:  0.9.0.dev-c697eeab84e5b8a74908da654b66ec9eca4f1291\n",
      "TensorFlow:  1.2.1\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using cuDNN version 5110 on context None\n",
      "Mapped name None to device cuda0: Tesla K80 (FC0D:00:00.0)\n"
     ]
    }
   ],
   "source": [
    "import keras\n",
    "print('keras: ', keras.__version__)\n",
    "\n",
    "# optional\n",
    "import theano\n",
    "print('Theano: ', theano.__version__)\n",
    "\n",
    "import tensorflow as tf\n",
    "print('TensorFlow: ', tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<h1 style=\"text-align: center;\">If everything worked till down here, you're ready to start!</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
